Skip to content

feat(opa-proto): support proto-format plan bundles (opa build --format=proto) - #110

Merged
sspaink merged 11 commits into
open-policy-agent:mainfrom
sspaink:worktree-proto-plan-bundles
Jul 30, 2026
Merged

feat(opa-proto): support proto-format plan bundles (opa build --format=proto)#110
sspaink merged 11 commits into
open-policy-agent:mainfrom
sspaink:worktree-proto-plan-bundles

Conversation

@sspaink

@sspaink sspaink commented Jul 8, 2026

Copy link
Copy Markdown
Member

Adds support for reading protobuf-format plan bundles produced by opa build --format=proto (plan.pb / .manifest.pb), decoding them into the existing IR model so they evaluate identically to their JSON counterparts.

Proto bundles keep data.json as JSON, so proto support is an optional add-on alongside a JSON provider (e.g. opa-jackson), not a replacement.

Closes #101

opa build --format=proto isn't in a tagged OPA release yet, so go.mod is pinned to a temporary commit (f6092b9ce, ~v1.19.0-dev; PRs open-policy-agent/opa#8825 and #8775).

Comment thread .github/workflows/pull-request.yml Fixed
Comment thread .github/workflows/pull-request.yml Fixed
Comment thread .github/workflows/pull-request.yml Fixed
Comment thread .github/workflows/pull-request.yml Fixed
Comment thread .github/workflows/pull-request.yml Fixed

@johanfylling johanfylling left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment thread cli/build.gradle.kts Outdated
Comment thread opa-proto/src/main/proto/README.md Outdated
Comment thread opa-proto/src/main/proto/v1/ir/plan.proto
@sspaink
sspaink force-pushed the worktree-proto-plan-bundles branch from 28999b7 to 34eb669 Compare July 24, 2026 16:27

@johanfylling johanfylling left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

sspaink added 11 commits July 30, 2026 16:10
…t=proto)

Add a new opa-proto module that decodes protobuf-format plan bundles
(plan.pb / .manifest.pb) into the existing IR model, so bundles built with
`opa build --format=proto` evaluate identically to their JSON counterparts.
Proto bundles keep data.json as JSON, so opa-proto is an optional add-on
alongside a JSON provider rather than a replacement.

opa-proto module:
- Vendored plan.proto / manifest.proto (Java bindings generated via
  protobuf-gradle-plugin + protobuf-java 4.x).
- PlanMapper (all 34 IR statement types), ManifestMapper, and StructConverter.
- ProtoBundleReader implementing the new optional ProtoBundleDecoder SPI.

opa-evaluator:
- ProtoBundleDecoder SPI (optional, zero-or-one).
- BundleFormat: filename constants + mixed-format rejection matching OPA.
- InputStreamSource + BundleAssembler.loadPlanAndManifest: shared
  detect/validate/route logic so loaders only do file discovery.
- Services.loadAtMostOne: one shared ServiceLoader helper (dedups three copies).
- PROTO_DECODER resolved lazily so proto misconfiguration cannot break
  JSON-only bundle loading.

Loaders/CLI:
- FileSystemBundleLoader and TarballBundleLoader detect .pb and route through
  the shared path.
- CLI depends on opa-proto at runtime so it can read proto bundles.

Version pinning:
- Pin the OPA version in tools/generate-compliance-tests/go.mod (single source
  of truth) and re-vendor .proto via the native vendorProtoSchemas Gradle task;
  CI verify-proto-vendor guards against drift.

Tests: proto-vs-JSON evaluation parity, mixed-format rejection, manifest
omitempty fidelity, numeric-type parity, and malformed-plan handling.

Closes open-policy-agent#101

Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
…ndor

Address zizmor findings on the verify-proto-vendor job: pin all four
actions to commit SHAs (blanket-pinning policy) and set
persist-credentials: false on checkout, matching the rest of the workflow.

Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
- Drop the misleading '.manifest.pb before .manifest' comment; branches
  use exact equals() matching, so order is irrelevant.
- Replace the redundant 'else if (!entry.isDirectory())' with a plain
  else; directories already continue at the top of the loop.

Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
Replace the lazy ProtoDecoderHolder with a static final PROTO_DECODER
field, matching POLICY_READER and BUNDLE_PARSER. loadOptional still
returns null for JSON-only classpaths, so the only behavioral change is
that a duplicate-provider misconfig now fails at class-init rather than
on first proto load — a classpath error we want to surface early.

Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
Per review: fold the three StructConverter number-conversion cases into a
@ParameterizedTest, and the two ProtoBundleParity per-entrypoint methods
into one parameterized test over (entrypoint, input). Structurally
distinct cases are left as their own @test methods.

Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
The other dependencies carry no rationale; opa-proto's purpose is clear
from the module name and documented in ProtoBundleDecoder's javadoc.

Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
ProtoBundleParityTest only exercised the proto decoder on one authz
bundle. This extends coverage to the full OPA compliance corpus so the
decoder is validated against the same breadth of IR the JSON reader is.

- generate-compliance-tests: emit a base64 `plan_proto` per case next to
  the JSON `plan`, produced by OPA's own ir.PolicyToProto (the same path
  as `opa build --format=proto`), so the proto fixtures are faithful to
  OPA rather than a Java-side re-encoding.
- ProtoComplianceTest: for every case, decode both the JSON and proto
  plans and assert they evaluate identically. Evaluation parity (not a
  structural Policy comparison) is the contract that matters — the JSON
  reader and proto decoder legitimately differ on how absent repeated
  fields are represented (null vs empty list), which does not affect
  evaluation.
- Regenerate the fixtures from the pinned OPA version so every case
  carries the new plan_proto column; the JSON content is unchanged.
- opa-proto: add jackson-databind as a test dependency for reading the
  fixtures.

Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
The opa-proto module was never wired into CI, so its tests (including the
new ProtoComplianceTest) never ran on PRs. Add a test-opa-proto job gated
by the change filter, extend the pr-check Rego (and its tests) with an
opa-proto rule, and add the job to the summary's required set.

opa-proto's filter also triggers on tools/generate-compliance-tests/
changes, since ProtoComplianceTest decodes the plan_proto fixtures that
generator emits (an OPA version bump there changes them).

Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
CodeQL flagged the leading String parameter of two parameterized tests as
unused (it fed only the {0} display name). Use it in the AssertJ
description instead, which resolves the finding and improves the failure
message.

Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
The test-opa-proto and verify-proto-vendor jobs were pinned before main's
latest Dependabot action bumps landed. Match the SHAs already used by the
other jobs so the workflow pins one version of each action.

Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
OPA v1.19.0 is the first tagged release with `opa build --format=proto`, and
main now pins it, so the commit-pin scaffolding can go:

- vendorProtoSchemas: drop the GOSUMDB bypass — a tagged release is in
  sum.golang.org, so the checksum database can stay on.
- proto/README.md: describe the pin as v1.19.0 rather than the interim commit.
- Services: restore the braced if body dropped while replaying onto main.

Signed-off-by: Sebastian Spaink <sebastianspaink@gmail.com>
@sspaink
sspaink force-pushed the worktree-proto-plan-bundles branch from 9524ed0 to 404514c Compare July 30, 2026 21:20
@sspaink
sspaink marked this pull request as ready for review July 30, 2026 22:16
@sspaink
sspaink requested a review from a team as a code owner July 30, 2026 22:16
@sspaink
sspaink merged commit bb79806 into open-policy-agent:main Jul 30, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support proto-format plan bundles (opa build --format=proto)

3 participants